AD-A221  784 


oiic . 


>  j 


!py 


ARI  Research  Note  90-09 


Evaluation  Plan  for  the  Gateway  2000 
Demonstration  Project 

David  R.  Hunter  and  Helen  V.  Lewis 


DTIC 

S  ELECT E  M 
APR2  6  1990  I  I 


Aviation  R&D  Activity  at  Fort  Rucker,  Alabama 
Charles  A.  Gainer,  Chief 

Training  Research  Laboratory 
Jack  H.  Hiller,  Director 


February  1990 


United  States  Army 

Research  Institute  for  the  Behavioral  and  Social  Sciences 

Approved  for  public  ’■eloio©;  distribution  is  unlimited. 


\ 

# 


U.S.  ARMY  RESEARCH  INSTITUTE 

FOR  THE  BEHAVIORAL  AND  SOCIAL  SCIENCES 


A  Field  Operating  Agency  Under  the  Jurisdiction 
of  the  Deputy  Chief  of  Staff  for  Personnel 


EDGAR  M.  JOHNSON 
Technical  Director 


JON  W.  BLADES 
COL.  IN 
Commanding 


Technical  review  by 

Charles  A.  Gainer 
Ronald  J.  Lofaro 
Gena  M.  Pedroni 


Accesio'i  f O’’ 

y 

NTIS  CR*AI 

LriC  TAd 

a 

O'dOOOi.  '.fti 

o 

By  . . 

Oisti  ibofioi’  1 

A 

vaildbil'fy  Codes 

Dist 

Avail  oi'd/or 

Special 

fi'l 

i 

NOTICES 

DISTRIDUTION:  This  report  has  been  cleared  for  release  to  the  Defense  Technical  Information 
Center  (DTIC)  to  comply  with  regulatory  requirements.  It  has  been  given  no  primary  distribution 
other  than  to  DTIC  and  will  be  available  only  through  DTIC  or  the  National  Technical 
Information  Service  (NTIS). 

FINAL  DISPOSITION:  This  reptort  may  be  destroyed  when  it  is  no  longer  needed.  Please  do  not 
return  it  to  the  U.S.  Army  Research  Institute  for  the  Behavioral  and  Social  Sciences. 


NOTE:  The  views,  opinions,  and  findings  in  this  report  are  those  of  the  auihor(s)  and  should  not 
be  consuued  as  an  official  Deparunent  of  the  Army  position,  policy,  or  decision,  unless  so 
designated  by  other  authorized  documents. 


V 


UNCLASSIFIED 


CURITY  CLASSIFICATION  OF  THIS  PAGE 


1«.  REPORT  security  CLASSIFICATION 

Unclassified 


2a.  SECURITY  CLASSIFICATION  AUTHORITY 


2b.  DECLASSIFICATION /CXJWNGRADING  SCHEDULE 


4.  PERFORMING  ORGANIZATION  REPORT  NUMBER(S) 
ARl  Research  Note  90-09 


REPORT  DOCUMENTATION  PAGE 


lb.  RESTRICTIVE  MARKINGS 


form  Approved 
0MB  No.  0704-0188 


3.  DISTRIBUTION /AVAILABILITY  OF  REPORT 
Approved  for  public  release; 
distribution  is  unlimited. 


5.  MONITORING  ORGANIZATION  REPORT  NUMBER(S) 


6a.  NAME  OF  PERFORMING  ORGANIZATION 
ARI  AVSCOM  Element 


6c  ADDRESS  {City,  State,  and  ZIP  Code) 

4300  Goodfellow  Blvd. 

St.  Louis,  MO  63120-1798 


6b.  OFFICE  SYMBOL 
(If  applicable) 

PERI-IRA 


7a.  NAME  OF  MONITORING  ORGANIZATION 
U.S.  Army  Research  Institute 
Aviation  R&D  Activity 


7b.  ADDRESS  (C/'ty,  State,  and  ZIP  Code) 

ATTN:  PERI-IR 

Fort  Rucker,  AL  36362-5354 


8b.  OFFICE  SYMBOL  i  9.  PROCUREMENT  INSTRUMENT  IDENTIFICATION  NUMBER 


8a.  NAME  OF  FUNDING /SPONSORING  8b.  OFFICE  SYMB 

ORGANIZATION  U.S.  Army  Research  (if  applicabtel 
Institute  for  the  Behavioral 
and  Social  Sciences  PERI-I 


8c  ADDRESS  (C/ty,  State,  aryd  ZIP  Code) 
5001  Eisenhower  Avenue 
Alexandria,  VA  22333-3600 


1 1 .  TITLE  (Include  Security  Classificdtion) 

Evaluation  Plan  for  the  Gateway  2000  Demonstration  Project 


12.  PERSONAL  AUTHOR(S) 

Hunter,  David  R.  and  Lewis,  Helen  V. 


13a.  TYPE  OF  REPORT 
Final 


10.  SOURCE  OF  FUNDING  NUMBERS 


PROGRAM 

PROJECT 

TASK 

WORK  UNIT 

ELEMENT  NO. 

NO. 

NO. 

ACCESSION  NO. 

63007A 

793 

1210 

HI 

13b.  TIME  COVERED 
FROM  88/05  TO  89/03 


14,  DATE  OF  REPORT  {Year,  Month.  Day)  15.  PAGE  COUNT 
1990,  February  39 


COSATI  CODES 


GROUP  SUB-GROUP 


J8.  SUBJECT  TERMS  (Continue  on  reverie  if  necessary  and  identify  by  block  number) 
Evaluation  Program  evaluation- 

Surveys  Questionnaires 

Employee  attitudes  Organizational  development.  / 


19.  ABSTRACT  {Continue  on  reverse  if  necessary  and  identify  by  block  number) 

The  (Gateway  2000  Demonstration  Project  is  designed  to  show  that  an  important  determinant 
of  effective  personnel  management  is  managerial  capacity  to  elicit  employee  involvement  in 
mission  accomplishment.  To  ascertain  the  validity  of  the  project’s  premise,  an  external 
contractor  will  conduct  a  multiyear  assessment  of  the  effectiveness  of  the  demonstration 
project  interventions  instituted  in  three  major  areas  of  personnel  management:  (1)  classifi¬ 
cation  and  compensation;  (2)  training  and  employee  development;  and  (3)  performance  evalu¬ 
ation  and  employee  recognition.  The  United  States  Army  Aviation  Systems  Command  and  the 
United  States  Army  Troop  Support  Command  located  at  the  Federal  Center  in  St.  Louis, 

Missouri,  will  participate  in  the  demonstration  project.  Located  near  Detroit,  Michigan, 
the  United  States  Army  Tank  and  Automotive  Command  will  serve  as  project  control  and  will 
receive  no  project  interventions.  This  evaluation  plan  specifies  the  minimum  requirements 
for  assessment  of  the  5-year  demonstration  project  and  provides  other  information  pertinent 
to  conducting  the  project  evaluation. 


20,  DISTRIBUTION /AVAILABILITY  OF  ABSTRACT 
G3  UNCLASSIFIED/UNLIMITED  □  SAME  AS  RPT. 


22a.  NAME  OF  RESPONSIBLE  INDIVIDUAL 
David  R.  Hunter 


00  Form  1473,  JUN  86  i 


21.  ABSTRACT  SECURITY  CLASSIFICATION 
□  DTIC  USERS  Unclassified 


22b  TELEPHONE  (Include  Area  Code)  22c.  OFFICE  SYMBOL 
(314)  263-2407  PERI-IRA 


Previous  editions  are  obsolete. 


SECURITY  CLASSIFICATION  OF  THIS  PAGE 
UNCLASSIFIED 


i 


EVALUATION  PLAN  FOR  THE  GATEWAY  2000  DEMONSTRATION  PROJECT 


CONTENTS _ 

Page 


INTRODUCTION  .  1 

Demonstration  Project  Overview .  2 

EVALUATION  DESIGN .  3 

Overview  and  Rationale .  3 

Implementation  Evaluation  .  3 

Experimental  Evaluation  .  4 

Summative  Evaluation .  4 

EVALUATION  PROCEDURE  .  5 

Measures  and  Data  Sources . . .  5 

Data  Collection  Schedule  and  Analysis  Plan .  8 

EVALUATION  MODEL  .  9 

Classification  Personnel  Management  Area .  10 

Performance  Evaluation  and  Employee  Recognition  ....  12 

Training  and  Employee  Development  .  14 

REFERENCES .  19 

APPENDIX  A.  PROJECT  DATABASE .  A-1 


x.jqr  OP  TABLES 

Table  1.  Summary  of  data  types,  sources,  and 


collection  frequency  .  6 

2 .  Evaluation  model  summary  for  personnel 

management  areas  .  16 

A-1.  G2K  database  system .  A-2 

A-2.  Data  sources  and  key .  A-14 


\ 


EVALUATION  PLAN  FOR  THE  GATEWAY  2000  DEMONSTRATION 

PROJECT 

INTRODUCTION 


The  Gateway  2000  Demonstration  Project  is  designed  to  show 
that  an  important  determinant  of  effective  personnel  management 
is  managerial  capacity  to  elicit  employee  involvement  in  mission 
accomplishment.  To  ascertain  the  validity  of  the  project's 
premise,  an  external  evaluation  contractor  will  conduct  a 
comprehensive,  multi-year  evaluation  of  the  effectiveness  of  the 
demonstration  project  interventions.  The  effectiveness  of 
interventions  instituted  in  three  areas  of  personnel  management 
will  be  assessed,  that  is:  (1)  classification  and  compensation; 

(2)  performance  evaluation  and  employee  recognition;  and 

(3)  training  and  employee  development.  The  United  States  Army 
Aviation  Systems  Command  and  the  United  States  Army  Troop  Support 
Command  (AVSCOM-TROSCOM)  comprise  the  demonstration  project 
experimental  site.  The  United  States  Army  Tank  and  Automotive 
Command  (TACOM)  will  serve  as  the  project  control  site,  and  will 
receive  no  project  inter-ventions.  This  evaluation  plan  specifies 
the  minimum  requirements  for  the  evaluation  of  the  five-year 
demonstration  project,  and  provides  other  information  pertinent 
to  the  conduct  of  the  multi-year  evaluation. 

The  plan  contains  a  brief  overview  of  the  demonstration 
project. ,  The  project's  purpose,  goals,  and  interventions  are 
detailed,  and  information  is  provided  about  the  project 
experimental  site  (AVSCOM-TROSCOM)  and  control  site  (TACOM) . 

The  evaluation  plan  provides  descriptive  information  and 
other  guidance  about  the  project's  evaluation  design  (quasi- 
experimental ,  multiple  time-series  design)  and  the  project  phase 
evaluations  (implementation,  experimental,  and  summative  project 
phase  evaluations) .  Contractual  documentation  requirements  for 
the  project  phase  evaluations  are  detailed. 

The  plan  also  contains  procedural  guidance.  It  specifies 
quantitative  measures,  data  sources,  and  instrumentation. 
Information  is  provided  about  the  data  collection  schedule  and 
data  analysis.  In  addition.  Appendix  A  of  the  evaluation  plan 
provides  detailed  information  about  the  project's  data  management 
procedures,  that  is,  information  about  the  project's  database 
structure,  content,  development,  and  maintenance. 

In  a  final  section  of  the  evaluation  plan,  a  provisional 
evaluation  model  is  applied  to  the  three  general  areas  of 
personnel  management.  The  objective  of  the  model  applications  is 
to  provide  more  definitive  guidance  concerning  the  implementation 
of  the  evaluation. 
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Demonstration  Project  Overview 

The  purpose  of  the  Gateway  2000  Demonstration  Project  is  to 
demonstrate  that  effective  personnel  management  depends,  in  large 
measure,  on  the  individual  manager's  capabilities  to  elicit 
employee  involvement  in  the  accomplishment  of  the  assigned 
mission.  The  demonstration  project  is  designed  to  be  in  place 
for  a  period  of  five  years.  Project  interventions  will  be 
instituted  for  approximately  5,000  employees  of  the  United  States 
Army  Aviation  Systems  Command  (AVSCOM)  and  the  United  States  Army 
Troop  Support  Command  (TROSCOM) .  The  AVSCOM-TROSCOM  activities, 
located  at  the  Federal  Center  in  Saint  Louis,  Missouri,  comprise 
the  demonstration  project  experimental  site.  A  control  site, 
that  receives  no  project  interventions,  will  be  used  to  assess 
the  effects  of  the  interventions  instituted  at  the  AVSCOM- 
TROSCOM. 

Located  near  Detroit,  Michigan,  the  United  States  Army  Tank 
and  Automotive  Command  (TACOM)  will  serve  as  the  demonstration 
project  control  site.  To  ensure  the  greatest  generalization  of 
the  results  of  the  evaluation  of  the  Gateway  2000  project,  the 
control  site  must  be  as  similar  as  possible  to  the  experimental 
site.  Several  other  major  subordinate  commands  of  the  United 
States  Army  Materiel  Command  perform  functions  similar  to  those 
of  the  AVSCOM-TROSCOM,  However,  based  upon  the  degree  of 
similarity  in  size,  structure,  mission  activities,  and  economic 
and  geopolitical  environments,  the  TACOM  was  judged  most  similar 
to  the  AVSCOM-TROSCOM. 

Some  of  the  specific  goals  of  the  demonstration  project  are 
to:  (1)  improve  communication  between  employees  and  managers; 

(2)  enhance  the  compensation  of  high  performers;  (3)  improve  the 
training  of  employees  for  specific  jobs;  (4)  provide  customized 
career  enhancement;  and  (5)  retain  a  quality  work  force  by  estab¬ 
lishing  performance  as  a  major  factor.  These  improvements  are 
expected  to  result  from  the  individual  and  combined  effects  of 
instituting  interventions  at  the  experimental  site  in  three 
general  areas  of  personnel  management:  (1)  classification  and 
compensation;  (2)  performance  evaluation  and  employee  recog¬ 
nition;  (3)  and  training  and  employee  development. 

Project  interventions  that  will  be  instituted  at  the  AVSCOM- 
TROSCOM,  the  experimental  site,  include;  (1)  delegation  of 
personnel  classification  authority;  (2)  creation  of  career  paths; 

(3)  implementation  of  pay  banding;  (4)  improved  performance 
appraisal;  (5)  decentralization  of  training  approval  authority; 
(6)  establishment  of  a  comprehensive  bonus  system;  and,  (7) 
development  of  a  program  to  assist  employees  obtain  college 
degrees . 
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Overview  and  Rationale 


A  quasi-experimental ,  multiple  time-series  design  (Campbell 
&  Stanley,  1963)  will  be  used  to  conduct  the  multi-year 
evaluation  of  the  effectiveness  of  the  Gateway  2000  project 
interventions.  During  the  demonstration  project,  periodic 
comparisons  will  be  made  of  changes  at  the  experimental  site 
(demonstration  period)  with  prior  states  at  the  experimental  site 
(pre-demonstration  period) .  Also,  comparisons  will  be  made  of 
changes  at  the  experimental  site  with  changes  at  the  control  site 
(demonstration  period) .  This  demonstration-versus-baseline, 
experimental-versus-control  group  framework  will  separate  the 
effects  of  the  demonstration  project  interventions  from  changes 
not  related  to  the  Gateway  2000  Demonstration  Project. 

Furthermore,  since  the  proposed  demonstration  project 
involves  multiple  changes  in  more  than  one  area  of  personnel 
management,  a  careful  evaluation  of  all  project  phases  will  be 
necessary.  Therefore,  the  multi-year  evaluation  of  the  project 
inventions  will  include  an  implementation,  experimental  and 
summative  evaluation  (named  to  correspond  with  the  project  phase 
in  which  the  evaluation  occurs) .  The  overall  time-series 
evaluation  design  will  be  implemented  in  the  context  of  this 
multi-phase  evaluative  structure.  Descriptions  of  each  project 
phase  evaluation  are  provided  below,  to  include  additional 
rationale  and  specific  guidelines  for  implementing  the  project 
phase  evaluations. 


Implementation  Evaluation 

An  evaluation  of  the  project  implementation  is  necessary  to 
verify  that  the  project  is  implemented  as  designed  and  that  the 
stated  processes  are  stable  and  operational.  Evaluations  of  the 
breadth  and  depth  of  the  training  provided  to  supervisors  in  the 
new  personnel  management  procedures  are  particularly  critical . 

In  addition,  it  is  important  that  assessments  be  made  of  the 
degree  of  employee  understanding  of  the  new  procedures  and 
project  goals.  Within  30  days  after  implementation  of  the 
project  interventions,  the  evaluation  contractor  will  provide  an 
initial  assessment  and  report  on  the  status  of  the  implementa¬ 
tion.  The  results  of  this  initial  evaluation  may  be  used  to 
modify  training  and  the  intervention  implementation  strategies, 
or  to  identify  additional  approaches  necessary  to  ensure  a 
satisfactory  implementation. 

At  the  end  of  the  project  implementation  phase,  a  report 
that  documents  all  activities  of  this  phase  will  be  submitted  by 
the  external  evaluation  contractor  to  the  project  officials  and 
to  the  Office  of  Personnel  Management  (0PM) .  This  report  will 
detail  all  aspects  of  the  project  implementation  phase  and  will 
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provide  additional  suggestions  on  how  to  improve  the  implementa¬ 
tion  of  the  Gateway  2000  project,  as  appropriate.  In  addition, 
the  report  will  include  an  assessment  of  project  training.  Any 
changes  made  to  the  intervention  design  during  the  implementation 
phase  will  be  carefully  documented  in  this  report,  to  include  the 
rationale  for  any  changes  made.  Also,  the  revised  design  will  be 
contained  in  the  implementation  phase  report. 


Experimental  Evaluation 

The  evaluation  of  the  quasi-experimental  phase  will  begin  as 
soon  as  the  project  is  determined  to  be  operational  and  stable  by 
the  project  staff,  who  will  be  advised  by  the  evaluation  contrac¬ 
tor.  Data  collection  initiated  prior  to  implementation  will 
continue  on  a  periodic  basis,  as  specified  (See  Table  1) . 

Reports  that  provide  information  about  the  progressive  effects  of 
the  various  inteirventions  will  be  submitted  by  the  external 
evaluator  at  least  on  an  annual  basis.  Jointly  these  reports 
should  provide  a  chronological  record  of  the  developmental 
history  of  the  experimental  phase  of  the  demonstration  project 
and  should  effectively  capture  trends  and  other  cyclic  variations 
surrounding  the  effects  and  the  stability  of  the  various  inter¬ 
ventions  at  the  experimental  site.  These  multiple  experimental 
phase  reports  will  be  submitted  to  project  officials  and  the 
Office  of  Personnel  Management  (0PM) . 


Summative  Evaluation 


At  the  conclusion  of  the  experimental  phase  of  the  project, 
an  overall  assessment  of  the  combined  effects  of  the  demonstra¬ 
tion  project  interventions  will  be  made.  Impacts  of  specific 
system  changes  will  be  discussed  separately  and  in  combination. 
Cause-and-ef feet  relationships  will  be  established,  to  the  extent 
possible,  using  the  results  of  comparative  analyses  of  pre¬ 
intervention  and  post-intervention  data  as  well  as  the  results  of 
comparative  analyses  of  experimental  and  control  site  data.  The 
effectiveness  of  each  intervention  will  be  assessed.  Efforts 
will  be  made  to  include  an  assessment  of  unanticipated  positive, 
as  well  as  negative  effects.  A  final  project  report  will  be 
issued  by  the  external  evaluator  and  submitted  to  the  project 
officials  and  0PM. 
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EVALUATION  PROCEDURE 


Measures  and  Data  Sources 


The  types  of  measures  that  will  be  used  in  the  evaluation 
include;  (1)  measures  of  the  socioeconomic  environment;  (2)  work 
force  quality  indices;  (3)  measures  of  employee  attitudes;  and 
(4)  organizational  process  indices.  Exemplary  data  sources 
include:  (1)  the  personnel  system  database;  (2)  SF  Form  50 

(Personnel  Action) ;  (3)  Employee  Intake  Surveys;  (4)  Employee 
Exit  Surveys;  and,  (5)  the  records  of  the  Civilian  Personnel 
Office  (CPO) .  Table  1  provides  a  more  complete  listing  of  data 
types  and  data  sources.  In  addition,  all  measures  and  data 
sources  identified  for  use  in  the  evaluation  are  provided  in 
Appendix  A,  which  describes  the  project's  data  management 
procedures.  An  overview  of  selected  measures  and  information 
about  their  use  is  provided  below. 

Measures  of  the  socioeconomic  environment  of  the  experimental 
and  control  sites  will  be  used  to  assess  the  comparability  of  the 
experimental  and  control  sites,  and  as  covariates  during  the 
analyses  of  the  intervention  effects.  Work  force  quality 
measures  will  allow  an  evaluation  of  the  differential  effects  of 
the  interventions,  and  will  provide  a  linkage  to  a  nationally 
normed  reference  group.  Extensive  measures  will  be  taken  of  the 
incoming  and  exiting  employees  to  assess  their  reasons  for 
choosing  to  work  at  (or  leave)  the  experimental  and  control 
sites,  and  to  allow  the  evaluation  of  the  effects  of  the 
interventions  on  various  categories  of  work  force  attrition. 

Attitude  surveys  will  be  administered  to  assess  the  impact 
of  the  interventions  on  various  employee  perspectives,  such  as 
employee  job  satisfaction;  intent  to  turnover  among  the  work 
force;  and  understanding  of  and  satisfaction  with  the  Gateway 
2000  project. 

A  large  number  of  organizational  process  measures  will  be 
taken  from  the  operation  of  the  Civilian  Personnel  Office  and 
other  staff  offices.  These  measures  will  be  used  to  assess  the 
impact  of  the  interventions  on  the  operation  of  the  personnel 
office  and  upon  such  areas  as  the  suggestion  program  and  Equal 
Employment  Opportunity  complaint  rates.  (A  crosswalk  of  specific 
interventions  and  their  associated  measures  is  provided  below  in 
the  evaluation  model  applications.) 
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Table  1. 


Summary  of  Data  Types, 

Sources,  and 

Collection  Frequency 

Data  Type 

*  Sources 

Collection  Frequency 

Employee  Demographics 

A/B 

Upon  Employee  Action 

Employee  Position 

A/B/C 

Upon  Employee  Action 

Employee  Performance 

A/C/D 

At  6-month  and  yearly 
evaluations 

Employee  Intake/Exit 

A/E/F 

Upon  entry  or  separation 

Work  force  Quality 

A/G/H/I/J/K 

One-time  sampling  of 
experimental  and  control 
groups.  Continual  test 
of  entry  employees. 

Managers  as  take  part  in 
assessment  process. 

Employee  Attitudes 

A/L 

Every  6  months,  total 
control  and  experimental 
group . 

Assessment  Process  | 

Indices 

A/M 

To  be  determined. 

Organizational 

Indicators 

Q 

Quarterly  I 

Socioeconomic  I 

Indicators 

N/P 

Quarterly 

*  Source  code  designations: 

A  Initial  exercise  of  the  Personnel  System  Database 
B  SF  FORM  50  (Personnel  Action) 

C  Previous  G2K  Database  entry  (Current  position  becomes 
previous  position  upon  receipt  of  SF  50  for  transfer) 
D  G2K  Personnel  Evaluation  Forms  (CPO) 

E  New  Employee  Intake  Survey 

F  Separating  Employee  Exit  Survey 

G  Employee  Quality  Survey 

H  Evaluator  Transforms  of  Quality  Indexes 

(Institutional  quality  ratings,  ratings  of  match 

between  major  field  of  study  and  job,  etc.) 
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Table  1  continued 


Summary  of  Data  Types,  Sources  and  Collection  Frequency 


*  Source  code  designations: 

Skills,  Knowledge  &  Abilities  Testing  Battery 
Management  Quality  Index  Test 

Professional  Level  Survey,  Professional  Societies 
and  Certification  Agencies,  and  Special  Testing. 
Employee  Attitude  Survey 

Assessment  Process 
Bureau  of  Labor  Statistics 
Local  Chamber  of  Commerce 
Civilian  Personnel  Office  Records 
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Data  Collection  Schedule  and  Analysis  Plan 


The  measures  reported  as  data  elements  in  Appendix  A  and 
summarized  in  Table  1  will  be  collected  at  the  experimental  and 
control  sites  prior  to  implementation  of  the  demonstration 
project  and  periodically  thereafter,  as  specified  in  Table  1. 

For  example,  socioeconomic  and  organizational  process  indicators 
will  be  collected  quarterly  throughout  the  life  of  the 
demonstration  project. 

Analysis.  Each  of  the  program  evaluation  analyses  will 
compare  changes  occurring  at  the  experimental  site  (demonstration 
period)  to  the  baseline  period  (pre-demonstration  period) ,  and 
also  will  compare  changes  at  the  experimental  site  (demonstration 
period)  with  changes  occurring  at  the  control  site.  In  addition, 
an  analysis  of  covariance  approach  may  be  utilized  to  assess  the 
extent  to  which  socioeconomic  environmental  factors  may  account 
for  the  observed  changes.  This  analytical  framework  will  permit 
the  differentiation  of  changes  that  occur  uniquely  at  the  experi¬ 
mental  site  from  other  systemic  changes  occurring  throughout  the 
Army  Materiel  Command,  the  Army  in  general,  or  society  at  large. 

The  statistical  methods  that  will  be  employed  to  conduct  the 
analyses  include:  (1)  multiple  linear  regression;  (2)  correla¬ 
tional  analysis;  (3)  factor  analysis;  (4)  analysis  of  variance; 
(5)  multivariate  analysis  of  variance;  (6)  analysis  of  covar¬ 
iance;  and  (7)  nonparametric  procedures,  such  as  chi-square. 
Exemplary  analyses  are  provided  below  in  the  applications  of  the 
provisional  evaluation  model  to  each  of  the  personnel  management 
areas  (See  Evaluation  Model) . 
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EVALUATION  MODEL 


Applications  of  a  provisional  evaluation  model  to  the  three 
general  areas  of  personnel  management  (i.e.,  classification  and 
compensation;  performance  evaluation  and  employee  recognition; 
and  training  and  employee  development)  are  described  below 
(Rutman  &  Mowbray,  1983) .  These  model  applications  are  presented 
to  provide  more  definitive  guidance  for  the  conduct  of  a 
comprehensive  evaluation  of  the  Gateway  2000  Demonstration 
Project.  The  components  of  the  evaluation  model  are:  (1) 
statement  of  the  problem;  (2)  objective;  (3)  hypothesis;  (4) 
interventions;  (5)  expected  effects  of  the  interventions;  (6) 
measures;  (7)  data  sources;  and  (8)  analysis (es) .  The  model 
applications  are  summarized  in  Table  2  that  lists  the 
interventions,  expected  effects,  and  measures  for  each  area  of 
personnel  management.  The  model  is  not  considered  to  be 
exhaustive  at  this  point  and  additions  and  adjustments  may  be 
made,  as  appropriate,  during  the  implementation  of  the 
evaluation. 

It  is  important  to  point  out  that  because  the  interventions 
will  be  implemented  as  a  set,  the  effects  of  any  single 
intervention  cannot  be  interpreted  as  though  it  were  implemented 
apart  from  other  interventions.  However,  each  intervention  does 
have  expected  measurable  effects,  some  of  which  are  specified  in 
the  model  applications. 
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Classification  Personnel  Management  Area 

Statement  of  Problem;  Today's  classification  system  encom¬ 
passes  more  than  400  different  white  collar  occupations  and  18 
grades  of  difficulty,  making  it  cumbersome  and  complex.  Many 
managers  and  employees  do  not  understand  and/or  accept  the  cur¬ 
rent  system.  This  system  of  18  distinct  grades  calls  for  rigid 
application  of  the  position  classification  standards.  This  rigid 
job  classification  and  the  resulting  over-specialization  too 
often  forces  managers  to  employ  more  manpower  than  is  actually 
required  to  assure  that  all  tasks  are  covered.  In  addition,  the 
lack  of  knowledge  and  the  lack  of  control  over  the  budget  pro¬ 
vides  little  incentive  for  managers/ supervisors  to  practice 
efficient  and  effective  position  management  in  their  organiza¬ 
tions. 

Objective:  Simplify  the  classification  system. 

Hypothesis;  Dissatisfaction  and  confusion  of  employees 
and  managers  and  inefficient  utilization  of  human  resources 
are  related  to  the  complexity  of  the  personnel  classification 
system. 

Interventions ; 

Delegate  classification  authority  along  with  a  civilian 
pay  ceiling  (CPC)  to  supervisors/managers. 

Establish  bands  which  combine  several  grades  into  one 
band. 

Replace  current  job  classification  standards  and  job 
descriptions  with  broad  level  descriptors  that  define 
"ranges"  of  work  within  a  given  level. 

Create  career  paths  that  contain  groups  of  occupations  or 
professions  that  are  similar  enough  to  warrant  similar 
treatment  in  compensation. 

Expected  Effects; 

Improved  manpower  utilization  for  mission  accomplishment. 

Improved  understanding  of  the  classification  system 
by  supervisory  and  non-supervisory  personnel. 

Change  in  the  image  of  the  Civilian  Personnel  Office  to  an 
"advisor"  organization. 

Increased  awareness  of  the  budget  process. 

Greater  employee  growth  and  development  opportunities. 
Increased  managers/supervisors  accountability. 
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Reduced  classification  time. 

Measures: 

Acceptance  ratio  (all  personnel) 

Attitudes 
-Job  Satisfaction 
-Career  opportunities 

-Civilian  Personnel  Office  function  and  utility 
-Classification  process 
-Organizational  effectiveness 
-Utilization  of  abilities  &  talents 
-Control  and  accountability  by  supervisors 
Number  of  personnel  actions 

Data  Sources: 

Civilian  Personnel  Office  transaction  records 

Attitude  surveys 

Analyses: 

Comparison  of  acceptance  ratios,  pre-intervention  and  post¬ 
intervention  with  the  control  site. 

Comparison  of  attitude  survey  responses,  pre-intervention 
and  post-intervention  with  the  control  site. 
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Performance  Evaluation  and  Employee  Recognition 

Statement  of  Problem;  Employees  perceive  no  direct  link 
between  performance  and  reward.  The  current  pay  system  has  no 
provision  for  providing  a  direct  link  between  performance  and 
pay.  Supervisors  use  promotion  and  incentive  awards  to  recognize 
some  deserving  employees.  Limitations  such  as  budget  constraints 
and  competitive  requirements  for  promotion  prevent  recognition  of 
all  deserving  employees.  Supervisors  and  employees  perceive  the 
current  performance  appraisal  process  as  a  paperwork  exercise, 
and  perceive  actual  performance  as  having  little  or  no  influence 
on  receiving  an  award.  These  perceptions  have  resulted  in 
poorly  developed  performance  standards  and  the  superficial  com¬ 
munication  that  exists  between  supervisors  and  employees.  The 
extensive  documentation  and  review  process  minimize  the  motiva¬ 
tional  connection  intended  since  recognition  is  so  far  separated 
in  time  from  actual  achievement. 

Objective:  Establish  a  direct  link  between  performance  and 

reward. 

Hypothesis:  The  linkage  of  pay  to  performance  is  related  to 

perceptions  of  equity  and  satisfaction  with  the  pay  and  perfor¬ 
mance  appraisal  system. 

Interventions : 


Institute  a  performance  appraisal  system  which  recognizes 
and  rewards  rapid  employee  development  in  new  positions  and 
sustained  performance  once  target  levels  are  achieved. 

Provide  frequent  initial  performance  evaluations  with  suc¬ 
cessful  performance  linked  to  base  pay  increases. 

Establish  a  bonus  system  to  serve  as  a  vehicle  for  recog¬ 
nizing  employee  performance. 

Lower  approval  levels  for  monetary  incentive  awards  (Special 
Acts  and  On-The-Spot  Awards) . 

Expected  Effects: 

Performance  objectives  perceived  as  more  meaningful. 

Improved  communication  between  supervisors  and  employees. 
Perception  of  a  direct  link  between  performance  and  award. 
Increased  number  of  employees  receiving  performance  awards. 
Increased  satisfaction  with  pay  and  bonuses. 

Reduced  paperwork  burden  for  managers  and  staff. 
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Measures; 


Attitudes 

Pay  and  bonus  satisfaction 
-  Performance  appraisal  system 

Frequency  and  effectiveness  of  communications 
Paperwork  burden  for  supervisors 
Data  Sources: 

Attitude  survey 
Analysis; 

Compare  attitude  pre-intervention  and  post-intervention 
with  control  site. 
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Training  and  Employee  Development 


Statement  of  Problem:  The  current  laws  and  regulations 
pertaining  to  the  training  of  federal  employees  interfere  with 
the  most  effective  means  of  providing  adequate  training  and 
development  to  the  total  work  force.  Both  the  Packard  Commission 
findings  and  the  work  force  survey  indicate  that  training  and 
development  could  be  delivered  in  a  more  effective  manner  if  the 
approval  process  was  decentralized,  the  current  regulations  and 
procedures  streamlined,  and  a  strengthened  process  for  identi¬ 
fying  training  needs  established.  The  work  force  survey  also 
strongly  suggests  that  pre-supervisory  training  should  be 
available  to  employees  prior  to  assuming  supervisory  positions 
and  that  the  prohibition  of  funding  for  degrees  does  not  serve 
the  best  interests  of  the  employee  or  the  organization. 

Objectives;  Develop  a  more  capable  and  motivated  work  force 
by  providing  training  and  development  opportunities  on  a  timely 
basis. 

Hypothesis;  The  training  provided  to  employees  contri¬ 
butes  to  the  overall  quality  and  efficiency  of  the  work  force. 
Additional  provisions  for  training  will  increase  the  retention  of 
high  quality  personnel  and  improve  recruitment  of  scientists  and 
engineers. 

Interventions ; 

Decentralize  the  approval  of  training  to  the  Director/Of fice 
Chief. 

Establish  an  assessment  process  to  determine  training  needs 
for  individual  employees  relating  to  supervisory  duties. 

Establish  a  Degree  Tuition  Program  to  reimburse  employees 
for  job  or  command  related  college  degrees. 

Implement  an  understudy  program  to  allow  for  competitive 
selection  of  an  employee  prior  to  the  departure  of  the 
incumbent . 

Expected  Effects; 

Improved  individual  training  and  development  plans. 

Increase  in  number  of  employees  pursuing  and  completing 
college  degrees. 

More  highly  qualified  applicants  for  employment. 

Improved  retention  of  high  quality  employees. 

Improved  perceptions  of  supervisory  effectiveness. 

Improved  perceptions  of  training  opportunities. 
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Measures; 


Attitudes: 

Training  opportunities 
Intent  to  turnover 

Supervisor  effectiveness  and  )cnowledge 
Managerial  perceptions  of  training  needs  and  benefits 
Quality  of  new  hires  and  separations. 

Educational  level  of  work  force 
Data  Sources: 

Civilian  Personnel  Office  records 
Attitude  survey 

Intake/Exit  Questionnaires  and  Assessments 
Analyses; 

Comparison  of  attitude  survey  items  and  scales  pre¬ 
intervention  and  post-intervention  with  control  group. 

Comparison  of  educational  levels  pre-intervention  and  post¬ 
intervention  with  control  group. 

Comparison  of  work  force  quality  indicators  and  intake/exit 
questionnaire  data  pre-intervention  and  post-intervention 
with  control  group. 


15 


Table  2 . 


Evaluation  Model  Summary  for  Personnel  Management  Areas* 


Expected 

Interventions 

Effects 

Measures 

Classification  Area 


Delegate  classifica¬ 
tion  authority 

Improve  understanding 
of  process 

Attitude 

Survey 

Improve  perception  of 
management  control 

Establish  pay  bands 

Improve  management 
flexibility 

Attitude 

Survey 

Improve  worker  job 
satisfaction 

Increase  supervisory 
accountability  and 
control 

Replace  current  job 

classification 

standards 

Increase  perceived 
career  flexibility 

Increase  management 
flexibility 

Attitude 

Survey 

Create  career  paths 

Increase  perceived 
career  flexibility 

Attitude 

Survey 

Improve  retention 

CPO  Database 

Performance  Evaluation 
Area 

Institute  a  pass/fail 
performance  evaluation 
system 

Improve  employee 
satisfaction 

Decrease  supervisor 
workload 

Attitude 

Survey 

Decrease  grievances 

CPO  Database 

Decrease  EEO/AA 
complaints 

EEO/AA  Office 
Records 
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Table  2  continued 


Evaluation  Model  Summary  for  Personnel  Management  Areas* 


Expected 

Interventions 

Effects 

Measures 

Link  performance 
to  pay 

Improve  satisfaction 
with  pay  and  rewards 

Attitude 

Survey 

Increase  perceived 
pay-performance  link 

Increase  retention  of 
high  performing 
employees 

Exit 

Survey 

Employee 

Quality 

Indicators 

Establish  bonus 
system 

Improve  satisfaction 
with  pay  and  rewards 

Attitude 

Survey 

Lower  award  approval 
levels 

Increase  perceptions 
of  supervisor  control 

Attitude 

Survey 

Trainina  and  Develooment 
Area 

Decentralize  training 
approval 

Improve  employee 
satisfaction 

Attitude 

Survey 

Establish  assessment 
process 

Earlier  training 
of  supervisors 

Attitude 

Survey 

Improve  relevance 
of  training 

Improve  perception 
of  career  opportunities 

Establish  Degree 

Tuition  Program 

Improve  work  force 
quality 

Improve  retention 

Work  Force 

Quality 

Indicators 

Implement  an 
understudy  program 

Improve  work  force 
quality 

Work  Force 

Quality 

Indicators 

Table  2  continued 

Evaluation  Model  Summary  for  Personnel  Management  Areas* 


*  Note;  The  effects  and  measures  listed  are  representative  of 
the  total  effects  and  associated  measures  for  the  proposed 
interventions,  and  therefore  should  only  be  considered  as  the 
minimum  a  priori  set.  Additional  effects  and  measures  will  be 
specified  by  the  evaluation  contractor  and  project  staff  prior  to 
implementation  and  during  the  course  of  the  project. 
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APPENDIX  A 


PROJECT  DATABASE 


In  order  to  conduct  the  long-term  evaluation  of  the  Gateway 
2000  demonstration  project  interventions,  continuous  control  of 
the  data  collected  during  the  life  of  the  project  is  essential. 
This  appendix  specifies  the  structure  of  the  project's  global 
database  (G2K  Database) ,  and  the  structure  and  content  of  the 
databases  that  comprise  the  global  database,  sub-databases.  For 
each  database,  the  document  indicates  the  data  elements  to  be 
collected,  data  sources  and  the  frequency  of  data  update  (as 
applicable) .  It  is  anticipated  that  these  databases  will  be 
managed  by  the  evaluation  contractor.  In  the  concluding  section, 
specific  guidance  is  provided  about  how  to  create  and  maintain 
the  database  as  well  as  other  information  pertinent  to  database 
integrity. 

Database  Structure  and  Content 


Depending  upon  the  specific  database  management  program 
selected,  some  minor  variation  in  the  database  structure  may 
occur.  The  following  specification  assumes  that  only  fixed 
record  lengths  are  supported  (as  for  example  in  DBase  III) . 
Therefore,  separate  records  must  be  maintained  for  recurring, 
variable  instance  data  elements  (such  as  surveys,  promotions, 
etc.).  Should  a  more  sophisticated  database  management  program 
be  available,  these  separate  records  may  be  combined.  In  any 
event,  the  overall  impact  of  these  considerations  should  be 
transparent  to  the  demonstration  project  staff,  and  the  U.  S. 

Ammy  Aviation  Systems  Command  (AVSCOM)  and  the  U.  S.  Army  Troop 
Support  Command  (TROSCOM)  management. 

The  global  database  will  henceforth  be  referred  to  as  the 
G2K  Database.  The  G2K  Database  will  consist  of  two  principal 
sub-databases,  i.e.,  G2KE  that  will  contain  data  collected  from 
the  experimental  site  (AVSCOM-TROSCOM)  and  G2KC  that  will  contain 
data  collected  from  the  Control  site  (TACOM) .  Each  sub-database 
(G2KE  and  G2KC)  is  further  subdivided  as  listed  in  Table  A-1. 


A-1 


Table  A-1. 


G2K  Database  System 

Database  Name 

Content 

size  rAoorox.) 

G2KX-EDEMO 

Employee  demographic 
information 

20  Fields 

200  Characters 

G2KX-EP0S 

Employee  position 
(Multi  data  blocks) 

10  Fields 

100  Characters 

G2KX-EPERF 

Employee  performance 
(Multi  data  blocks) 

10  Fields 

100  Characters 

G2KX-EIED 

Employee  intake/exit 
information 

20  Fields 

200  Characters 

G2KX-QUAL 

Work  force  quality  data 

50  Fields 

200  Characters 

G2Kx-ESURVn 

Employee  attitude  survey  200  Fields 
(Multiple  entries)  250  characters 

G2KE-AC 

Assessment  process  data 

200  Fields 

1000  Characters 

G2Kx-ORGn 

Organizational  process 
and  outcome  data 
(Multiple  entries) 

30  Fields 

300  Characters 

G2Kx-EXTn 

External  socioeconomic 
data 

40  Fields 

200  Characters 

Note.  X  will  be  either  E  for  AVSCOM-TROSCOM  or  C  for  TACOM 
Total  expected  size  of  database  after  five  years  is  40MB. 
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The  structure  and  content  of  each  database  is  specified 
below,  that  is,  the:  (1)  Employee  Demographic  and  Background 
Information  Database;  (2)  Employee  Position  Database;  (3) 

Employee  Performance  Appraisal  Database;  (4)  Employee  Intake  and 
Exit  Database;  (5)  Work  force  Quality  Index  Database;  (6) 
Employee  Attitude  Survey  Database;  (7)  Assessment  Process 
Database;  (8)  Organizational  Process  and  Outcome  Database;  and 
(9)  External  Socioecomonic  Database.  The  translation  of  the 
entries  in  the  source  columns  is  provided  in  Table  A-2  that  lists 
the  data  sources,  and  their  key  designations  (e.g. ,  "A” 
designates  the  data  source  "Initial  Exercise  of  Personnel  System 
Database") . 

G2KX-EDEM0:  Employee  Demographic  and  Background 

Information  Database 

Source  Description  Code(s) 

Identification  block: 

A  1.  Special  ID  Code 

Data  block: 


A/B 

2. 

Date  of  Birth 

A/B 

3. 

Service  Computation  Date 

A/B 

4  . 

Occupational  Group  Code 

A/B 

5. 

Veteran  Preference  Status 

A/B 

6. 

Handicap  Codes 

A/B 

7. 

Tenure  Status 

A/B 

8. 

Education  Level 

A/B 

9. 

Year  Education  Completed 

A/B 

10. 

Ethnic  Status 

A/B 

11. 

Gender 

A-3 


Source 


Code (s) 


G2KX-EP0S:  Employee  Position  Database 

(Current  and  Previous) 

Description 

Identification  block: 


A  1.  Special  ID  Code 

Data  block  (Current  Position) : 


A/B 

2. 

Date  Occupied  Position 

A/B 

3. 

Employing  Command 

A/B 

4. 

Job  Series  Number 

A/B 

5. 

Pay  System  (Demo;GS;GM) 

A/B 

6. 

Grade  or  Level  (GS/GM  01-15; 

Demo 

I-IV) 

A/B 

7. 

Pay  Step  of  Grade  (GS/GM  only) 

A/B 

8. 

Salary 

A/B 

9. 

Supervisory  Status 

Data  block  (Previous  Position) : 

B/C 

10. 

Date  Occupied  Position 

B/C 

11. 

Employing  Command 

B/C 

12. 

Job  Series  Number 

B/C 

13. 

Pay  System  (Demo;GS;GM) 

B/C 

14. 

Grade  or  Level  (GS/GM  01-15; 

Demo 

I-IV) 

B/C 

15. 

Pay  Step  of  Grade  (GS/GM  only) 

B/C 

16. 

Salary 

B/C 

17. 

Supervisory  Status 

Data  Block  (Previous  Position) ; 

B/C 

18. 

Date  Occupied  Position 

B/C 

19. 

Employing  Command 

B/C 

20. 

Job  Series  Number 

B/C 

21. 

Pay  System  (Demo;GS;GM) 

B/C 

22. 

Grade  or  Level  (GS/GM  01-15; 

Demo 

I-IV) 

B/C 

23. 

Pay  Step  of  Grade  (GS/GM  only) 

B/C 

24. 

Salary 

B/C 

25. 

Supervisory  Status 
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G2KX-EPERF;  Employee  Performance  Appraisal  Database 


Source  Description 


Code (s) 


A 


C 

C 


Identification  block: 

1.  Special  ID  Code 

Data  Block  (Last  Evaluation) : 

2 .  Date  of  Evaluation 

3.  Type  of  Evaluation  (1  =  6mo.  Demo;  2  =  yrly  Demo; 

3  =  Control  yrly) 

4.  Outcome  (Literal  for  control;  0/1  for  6mo  demo; 

1-3  for  yrly  demo) 

5.  Monetary  Result  (Salary  increase  for  6mo  demo  eval; 

bonus  for  yrly  demo; 
leave  blank  for  controls) 

Data  Block  (Previous  Evaluation) : 

6.  Date  of  Evaluation 

7.  Type  of  Evaluation  (1  =  6mo.  Demo;  2  =  yrly  Demo; 

3  «  Control  yrly) 

8.  Outcome  (Literal  for  control;  0/1  for  6mo  demo; 

1-3  for  yrly  demo) 

9.  Monetary  Result  (salary  increase  for  6mo  demo  eval; 

bonus  for  yrly  demo; 
leave  blank  for  controls) 


[  REPEAT  DATA  BLOCKS  25  TIMES  TO  ALLOW  FOR  POSSIBLE  NUMBER 
OF  EVALUATIONS  OVER  THE  FIVE  YEAR  LIFE  OF  PROJECT.  ] 
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G2KX-EIED;  Employee  Intake  and  Exit  Database 


Source 


A 


A 


E 


Description  Code(s) 

Identification  block: 

1.  Special  ID  Code 


Data  Block  (Intake) : 


2.  Hire  Status  (1  *  Existing  employee  at  demo  start; 

2  ==  Employee  hired  after  demo  start 
Remainder  of  Intake  block  blank  for  Hire  Status  =1) 


3 .  Source 


(1  =  Previously  unemployed/school; 

2  =  Employed  outside  federal  gov't; 

3  =  Transfer  from  outside  DoD; 

4  =  Transfer  from  AF  or  Navy; 

5  =  Transfer  from  AVSCOM/TROSCOM-TACOM; 

6  =  Transfer  from  other  Army) 


4. 


Intake 


Survey  Questionnaire  Items 


25. 

Data  Block  (Exit) : 


26.  Date  of  Separation 


27. 


Destination 

2 

3 

4 

5 

6 

7 

8 


(1  =  retirement  from  fed  service; 
=  unemployed/none  stated; 

=  school; 

=  outside  federal  gov't; 

=  Fed  Gov't,  outside  DoD 
=  DoD,  outside  Army 
=  AVSCOM/TROSCOM-TACOM; 

=  Other  Army) 


28.  Level  (1  =  promotion; 

2  =  lateral; 

3  =  reduction) 

29.  Exit  Survey  Questionnaire  Items 


40. 
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G2KX-0UAL:  Work  Force  Quality  Index  Database 


Source 


A 


G 

G 

G 

G 

G 

G 

G 

G 

G 

G 

H 


Description 
Identification  block: 

1.  Special  ID  Code 
Data  block:  (Education) 

2.  Years  of  schooling 

3.  Degree (s)  awarded 

4.  Date  of  degree (s) 

5.  Grade-point  average 

6.  Rank  in  Class 

7 .  Institution  Attended 

8.  Major  field  of  study 

9.  Continuing  education  quantity 

10.  Continuing  education  kind 

11.  Continuing  education  source 
12  -  20.  Reserved 

Data  Block:  (SKA  Test  Data) 

21.  Test  Score  1 


Code(s) 


40.  Test  Score  21 

Data  Block:  (Management  Quality  Index  for  Supervisors) 

41.  MQI  Scale  1 

• 

• 

50.  MQI  Scale  10 

Data  Block:  (Special  Qualification  Data) 

51.  Type  of  data  to  follow  (1  =  Engineering 
Certification  Score;  2  =  CPA  Score  etc.) 

52.  Qual  Score  1 
55.  Qual  Score  4 
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G2Kx-SURVn;  Employee  Attitude  Survey  Database 
(Database  repeated  n  tiroes) 

Source  Description  Code(s) 

Identification  block: 

A  1.  Special  ID  Code 

Data  Block: 

2.  Date  of  Survey  Administration 

3.  Survey  version  number 

4.  Questionnaire  Items 

• 

200. 
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G2KE-AC;  Assessment  Process  Database 


Source 

A 

M 

M 


Description  Code(s) 

Identification  block: 

1.  Special  ID  Code 
Data  Block: 

2 .  Date  of  Assessment 

Additional  Data  Elements  will  be  defined  during 
Assessment  Process  development. 
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G2Kx-0RGn;  Organizational  Process  and  Outcome  Database 

(Assuming  one  pre-implementation  data  collection  and  data 
collection  at  three  month  intervals  thereafter.  There  will  be  21 
copies  of  this  database,  numbered  sequentially  from  1  (the  pre¬ 
intervention  data)  to  21  (the  final  data  collection  of  the  five 
year  project) . 

Source  Description  Code(s) 

B/Q  Data  block:  (Population  Indices) 

1.  Actual  Strength 

2 .  Average  Strength 

3.  Number  GS  employees 

4 .  Number  GM  employees 

5.  Number  GS  female  employees 

6.  Number  GM  female  employees 

7.  Percentage  of  females  in  work  force 

8 .  Number  GS  minority  employees 

9.  Number  GM  minority  employees 

10.  Percentages  of  minorities  in  work  force 

11.  Number  civilian  supervisors  who  have 
been  supervisors  more  than  6  months 

12.  Number  military  supervisors  who  have 
been  supervisors  more  than  6  months 

B/Q  Data  Block:  (Classification) 

13.  Number  of  high  grades  (11-15) 

14.  Number  of  grades  (1-10) 

15.  Average  grade  for  command 

16.  Number  SES  positions 

17.  Number  upward  mobility  positions 

18.  Number  of  positions  reviewed 

19.  Number  of  positions  audited 

B/Q  Data  Block:  (Recruitment  and  Placement) 

20.  Number  of  GS-13  and  above  positions 
filled  by  grade 

21.  Number  of  GS-9  thru  12  positions  filled  by 
grade 

22.  Number  of  GS-5  thru  8  positions  filled  by  grade 

23.  Number  of  GS-1  thru  4  positions  filled  by  grade 

24.  Fill  rate  of  Upward  Mobility  positions 

25.  Number  of  handicapped  new  hires 
(referrals  and  selections) 

26.  Number  of  Severely  Handicapped  new  hires 

27.  Number  of  Disability  Veteran  new  hires  (DAVs) 

28.  Number  of  Upward  Mobility  employees  graduated 

29.  Number  of  Upward  Mobility  positions  under 
recruitment 
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30.  Number  of  days  from  CPO  receipt  SF  52  to 
commitment  of  position 

31.  Number  of  positions  committed  during  reporting 
period 

32.  Average  number  of  days  to  commit  position 

B/Q  Data  Block:  (Technical  Services) 

33.  Number  of  personnel  actions  processed  by 
quarter  (Reassignments,  realignment,  details, 
changes  in  organizational  name,  within-grade 
increases,  employee  name  and  address  changes, 
and  change-to-lower  grade) 

Q  Data  Block:  (Performance  Ratings) 

34.  Number  of  ratings  to  be  completed 

35.  Number  ratings  completed  within  45  days  of  end 
of  rating  period 

36.  Percentage  of  ratings  completed  within  45  days 

37.  Number  of  ratings  processed  which  were 
completed  in  less  than  75  days  after  end  of 
rating  period 

Q  Data  Block:  (Training) 

38.  Number  of  hours  of  training  per  quarter 

39.  Number  of  course  participants  per  quarter 

40.  Total  training  cost  by  quarter 

41.  Average  number  course  per  employee 

42.  Allocated  spaces;  spaces  used;  hours 

of  training  at  resident  training  schools. 

43.  Percentage  of  authorized  training  spaces  used 

44.  Number  of  executive  seminars 

45.  Number  of  executive  development  programs 

46.  Number  of  employees  in  executive  development 
programs 

47.  Number  of  civilian  supervisors  with  more  than 
6  months  on  job  who  have  received  the 
required  supervisory  training 

48.  Number  of  military  supervisors  with  more  than  6 
months  on  job  who  have  received  the  required 
supervisory  training 

49.  Percentage  of  supervisory  training 
accomplished  (military  and  civilian) 
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Data  Block:  (Awards) 


50.  Number  of  Quality  Increases  (QI's)  (total) 

51.  Number  of  QI’s  to  GS  employees 

52.  Number  of  QI's  to  females 

53.  Number  of  QI's  to  minorities 

54 .  Number  of  performance  awards  to  GS  and  GM 
employees  (total)  (QI's  and  Sustained  Superior 
Perfoirmance  Awards) 

55.  Number  of  performance  awards  to  GS  and  GM 
female  employees 

56.  Number  of  performance  awards  to  GS  and  GM 
minority  employees 

57 .  Number  and  amount  of  awards  by  grade 
and  directorate 

Q  Data  Block:  (Suggestions) 

58.  Number  of  suggestions  submitted 

59.  Number  of  suggestions  adopted 

60.  Number  of  suggestions  over  180  days  old 

61.  Number  of  suggestions  approved,  not  implemented 

62 .  Number  of  suggestions  under  evaluation 

63.  Suggestion  backlog  rate  (%) 

64.  Tangible  benefits  from  civilian 
suggestions  for  last  3  years 

65.  Percentage  of  suggestion  goal  achieved 

Q  Data  Block:  (Sick  Leave) 

1.  Number  of  sick  leave  hours  used 

2.  Average  sick  leave  used  per  employee 

Q  Data  Block:  (Management-Employee  Relations) 

1.  Number  of  written  reprimands 

2.  Number  of  suspensions 

3 .  Number  of  demotions 

4 .  Number  of  removals 

5.  Total  adverse  actions 

6.  Number  of  grievances  (Negotiated  vs  Department 
of  the  Army) 

7.  Number  Unfair  Labor  Practice  complaints  issued 
by  Federal  Labor  Relations  Agency 

Q  Data  Block:  (Miscellaneous) 

1.  Number  of  retirees 

2.  Number  of  retirement  eligibles  by  directorate 

3.  Number  of  on-the-job  injuries 
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G2Kx-XTLn;  External  Socioecomomic  Factors  Database 


(Data  collected  quarterly  and  n  varies  from  1,  pre¬ 
implementation,  to  21.) 

Source  Description  Code(s) 

N/P  Data  Block:  (Employment  Rates) 

1.  Overall  local  employment  rate 

2 .  Local  employment  rate  by  occupation 

3.  •• 

4.  " 

5.  " 

6.  •' 

7.  " 

8.  •• 

Data  Block:  (Work  Force  Demographics) 

9.  Total  available  labor  pool  in  metropolitan 
area 

10.  Male  percent  in  labor  pool 

11.  Female  percent  in  labor  pool 

12.  Minority  percent  in  labor  pool 

13.  Male  minority  percent  in  labor  pool 

14 .  Female  minority  percent  in  labor  pool 

15.  Average  educational  level  of  labor  pool 

16.  Labor  pool  available  by  occupation 

17.  ” 

18.  " 

19.  '• 

20.  " 

21.  " 

22.  •' 


Data  block:  (Economic  Conditions) 

23.  Median  yearly  wage 

24.  Median  yearly  wage  by  occupation 

25.  " 

26.  •' 

27.  ” 

28.  ” 

29.  " 

30.  •' 

31.  Median  new  home  selling  price 

32.  Cost  of  living  index 
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Table  A-2. 


% 


Data  Sources  and  Key 


Data  Source 


Initial  exercise  of  the  Personnel  System  Database 
SF  FORM  50  (Personnel  Action) 

Previous  G2K  Database  entry  (Current  position  becomes 
previous  position  upon  receipt  of  SF  50  for  transfer) 
G2K  Personnel  Evaluation  Forms  (CPO) 

New  Employee  Intake  Survey 
Separating  Employee  Exit  Survey 
Employee  Quality  Survey 
Evaluator  Transforms  of  Quality  Indexes 
(Institutional  quality  ratings,  ratings  of  match 
between  major  field  of  study  and  job,  etc.) 

I  Skills,  Knowledge  &  Abilities  Testing  Battery 

J  Management  Quality  Index  Test 

K  Professional  Level  Survey,  Professional  Societies 

and  Certification  Agencies,  and  Special  Testing. 

L  Employee  Attitude  Survey 

M  Assessment  Process 

N  Bureau  of  Labor  Statistics 

P  Local  Chamber  of  Commerce 

Q  Civilian  Personnel  Office  Records 
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Database  Creation  and  Maintenance 


Initial  Database  Generation;  At  six  months  prior  to 
implementation,  the  personnel  system  database  will  be  exercised 
to  produce  the  initial  loading  of  the  G2K  Database  System.  This 
exercise  will  create  each  of  the  employee  database  elements 
indicated  above  and  will  initialize  each  with  the  appropriate 
header  identification  blocks  and  the  entire  Employee  Demographics 
database  (G2KX-EDEM0) .  The  personnel  system  database  will  be 
exercised  again  at  the  completion  of  the  implementation  phase  to 
establish  the  work  force  in  place  at  the  start  of  the  project. 

New  records  will  be  created  for  those  employees  not  present 
during  the  first  exercise,  and  the  records  for  departed  employees 
will  be  purged. 

Database  Contents  Verification;  A  random  sample  of 
approximately  10%  of  the  total  employee  database  entries  of  the 
G2KX-DEM0  database  will  be  printed  and  forwarded  to  employees  for 
verification  of  the  data.  Should  the  error  rate  indicated  by 
this  exercise  exceed  1%  of  the  critical  data  elements  (i.e., 
special  identification  codes,  education,  and  service  computation 
date)  then  the  entire  contents  of  the  G2KX-EDEM0  database  will  be 
subjected  to  employee  review.  This  verification  exercise  will  be 
repeated  for  the  G2KX-EDEM0  and  other  databases  on  a  yearly 
basis. 


Database  Security;  One  secure  confidential  copy  of  the 
crosswalk  between  the  special  identification  code  and  the 
employee  identification  data  (social  security  number  and  name) 
will  be  maintained.  To  insure  conf idcntiality  of  employee 
responses,  access  to  this  information  will  be  rigidly  restricted. 


Database  Upkeep  Responsibility;  It  will  be  the 
responsibility  of  the  evaluation  contractor  to  maintain  the 
database.  For  the  most  part,  data  for  entry  into  the  G2K 
Database  System  will  come  from  the  normal  records  which  are 
processed  through  the  personnel  office  (i.e..  Form  50's,  etc.). 
The  evaluation  contractor's  resident  coordinator  must,  therefore, 
be  formally  placed  in  the  chain  of  distribution  for  all  personnel 
actions  and  other  relevant  documents.  It  will  be  the 
coordinator's  responsibility  to  extract  the  required  information, 
code  it  appropriately,  and  enter  it  into  the  database.  To  ensure 
database  integrity,  stringent  controls  must  be  enforced  by  the 
evaluation  contractor  on  the  resident  coordinator,  and  extensive 
quality  control  checks  must  be  built  into  the  database  update 
process.  In  addition,  appropriate  safeguards  against  data  loss 
must  be  taken  through  a  regular  and  strictly  enforced  system  of 
transaction  journals  and  storage  media  backup.  Provisions  for 
these  and  other  procedures  will  be  built  into  the  statement  of 
work  for  the  evaluation  contract. 
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It  is  absolutely  essential  that  complete  and  equivalent  data 
be  collected  from  both  the  experimental  and  control  sites.  It  is 
anticipated  that  the  evaluation  contractor  will  place  a  full  time 
employee  (resident  data  coordinator)  at  both  the  experimental  and 
control  sites  to  ensure  data  integrity.  The  resident  data 
coordinator  will  conduct  the  day-to-day  data  collection 
activities,  including  administration  of  the  Intake,  Quality 
Assessment  and  Exit  Questionnaires;  and  to  the  extent  possible, 
the  coordinator  will  relieve  the  personnel  office  of  any  burden 
related  to  the  data  collection  for  this  evaluation. 

Other  Considerations;  Because  of  the  importance  placed  upon 
employee  turnover  and  its  control  under  this  project,  valid  data 
regarding  the  reasons  for  turnover  are  vital.  Previous 
experience  has  shown  that  employees  are  sometimes  unwilling  to 
candidly  state  their  reasons  for  leaving  or  accepting  a  job,  for 
fear  of  the  impact  on  future  personnel  actions.  Therefore,  it  is 
proposed  that  during  the  course  of  this  project  the  evaluation 
contractor  will  assume  responsibility  for  all  entry  and  exit 
questionnaires  with  assurances  of  complete  anonymity  provided  to 
the  employees.  This  action  will  hopefully  increase  the  validity 
of  the  information  obtained.  Further,  the  contractor  will  be 
instructed  to  vigorously  follow-up  to  ensure  that  all  departing 
employees  and  new  employees  complete  the  survey  instruments. 
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